Stabilization of Tool-Carrying End of Extended-Reach Arm of Automated Apparatus

ABSTRACT

Apparatus and methods that can be used to stabilize the distal end of an arm (and an end effector attached thereto) of an automated extended-reach tool-equipped assembly. Stabilization is provided by three or more stabilizers, each comprising a stationary part and a movable part. Each stationary part has a fixed location relative to the end effector; each movable part is translatably coupled to a respective stationary part and comprises a contactor disposed at a distal end of the movable part. When the stabilizers are actuated, the contactors are translated toward and into contact with the surface of the workpiece and then locked in place to stabilize the distal end of the arm and the end effector. During tool operation, the stabilizers reduce oscillation of the end effector (and all structure fixedly coupled thereto).

RELATED PATENT APPLICATION

This application is a continuation-in-part of and claims priority fromU.S. patent application Ser. No. 15/623,304 filed on Jun. 14, 2017.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

The invention described herein was made in the performance of workpartially funded under NASA Contract No. NASA ACC, 2C10-2.2.2,WA-16-01340 and is subject to the provisions of Section 305 of theNational Aeronautics and Space Act of 1958 (72 Stat. 435: 42 U.S.C.2457).

BACKGROUND

This disclosure relates to systems and methods for fabrication, repairand maintenance of large structures (such as aircraft) using a toolmounted on the end of an arm of an automated apparatus.

SUMMARY

The subject matter disclosed herein is directed to apparatus and methodsthat can be used to stabilize the distal end of an arm (and an endeffector attached thereto) of an automated extended-reach tool-equippedassembly to enable the dimensional mapping, non-destructive inspection(NDI) and/or repair of large structures such as aircraft. (As usedherein, the term “end effector” means the last link of an automatedapparatus comprising an arm, at which endpoint a maintenance tool isattached.) The end effector is capable of holding and orienting a probeor sensor (e.g., NDI sensor unit, laser scanner, camera, etc.) or othertool (e.g., drill, sander, vacuum, “grabber”, coating sprayer, etc.).The stabilization technology disclosed herein reduces oscillation of theend effector (and tool attached thereto) relative to a target object,while still allowing for a rapidly deployable on-ground robotic mobileplatform. The stabilization devices disclosed herein can also be used toreduce load on robotic arm motors, which improves the power useefficiency and lifespan of those motors.

In accordance with various embodiments, stabilization is provided bythree or more stabilizers each comprising a stationary part and amovable part. Each stationary part has a fixed location relative to anend effector; each movable part is translatably coupled to a respectivestationary part and comprises a contactor disposed at a distal end ofthe movable part. When the stabilizers are actuated, the contactors aretranslated toward and into contact with the surface of the workpiece andthen locked in place to stabilize the distal end of the arm and the endeffector coupled thereto. During tool operation, the stabilizers resistmovement of the end effector (and all structure fixedly coupled thereto)in a direction substantially parallel to the axes of the stabilizers(which may be mutually parallel) and toward the workpiece surface.

In accordance with some embodiments, each stabilizer comprises atelescopic tube and a spring that urges a contactor (e.g., a contact pador bumper) against the surface of the workpiece. In accordance withother embodiments, each stabilizer comprises a pneumatic cylinder thaturges a contactor against the surface of the workpiece. The stabilizersgenerate forces at the contact locations substantially perpendicular tothe stabilizers to resist motion relative to the surface. Anotherapproach for stabilization is to provide devices that attach to thesurface of the workpiece by attraction, adhesion or gripping. Forexample, in accordance with various embodiments, the stabilizationdevices may employ suction, electrostatic attraction, releasableadhesive (like the strips used to mount removable hooks on walls), or a“gripper” that physically grabs a protuberance on the workpiece.

The technology disclosed herein enables modular, rapid, low-cost roboticuse for aircraft and other large structure applications by providingstabilization at the distal end of an arm of an extended-reach automatedapparatus. The disclosed technology overcomes one or more of thedisadvantages which accompany some techniques for extending a roboticarm to a working location while maintaining positional precision, whichdisadvantages include excessive weight, complexity and cost and lack ofmodularity.

In accordance with one NDI application, a ground-crawling automatedapparatus has an end effector configured to accept interchangeable NDIdevices, such as an infrared thermography scanner and an ultrasonictransducer array scanner. In accordance with another NDI application,the automated apparatus has an end effector configured to accept aninfrared thermography scanner having an ultrasonic transducer arrayscanner movably coupled thereto. In the first application, eitherscanner, and in the second application, the combined scanners can bestabilized using the apparatus disclosed herein. The ability tostabilize the distal end of an extended-reach arm enables the use oflow-cost extended-reach robots for aircraft fabrication, repair andmaintenance.

Although various embodiments of apparatus and methods for stabilizing anend effector coupled to a distal end of an extended-reach arm of anautomated apparatus will be described in some detail below, one or moreof those embodiments may be characterized by one or more of thefollowing aspects.

One aspect of the subject matter disclosed in detail below is anautomated apparatus comprising: an arm having a distal end; an endeffector which is coupled to a distal end of the arm; a first tool whichis coupled to the end effector; and a plurality of stabilizers eachcomprising a stationary part and a movable part, wherein: eachstationary part has a fixed location relative to the end effector; andeach movable part is translatably coupled to a respective stationarypart and comprises a contactor disposed at a distal end of the movablepart. Each contactor may comprise a fixed bumper or a pivotable contactpad. In either case, the contactors are preferably made of elastomericmaterial.

In accordance with one embodiment of the automated apparatus describedin the preceding paragraph, each stabilizer of the plurality ofstabilizers further comprises a spring, and for each stabilizer of theplurality of stabilizers: the stationary part comprises an outer tube;the movable part further comprises an inner shaft slidably disposedinside the outer tube; the contactor is attached to a distal end of theinner shaft; and the spring urges the inner shaft to translate in adirection so that the contactor moves further away from the endeffector.

In accordance with another embodiment, for each stabilizer of theplurality of stabilizers: the stationary part comprises a base cylinder;the movable part further comprises a piston that is slidable inside thebase cylinder and a piston rod connected to the piston; the contactor isattached to a distal end of the piston rod; and the stationary partfurther comprises a rod lock attached to the base cylinder andconfigured for locking the piston rod to prevent movement.

Another aspect of the subject matter disclosed in detail below is amethod for performing a maintenance operation, comprising: (a) moving arobotic mobile platform to a first location where a tool at a distal endof an arm is able to reach a first area of interest on a surface of atarget object; (b) configuring the robotic mobile platform such that anend effector is at a correct location relative to the first area ofinterest; (c) extending contactors of a plurality of stabilizers intocontact with the surface of the target object, which stabilizers arecoupled to the arm; (d) locking the contactors of the stabilizers inplace to maintain the end effector at the correct location relative tothe first area of interest; and (e) performing a first maintenanceoperation using the tool while the end effector is maintained at thecorrect location relative to the first area of interest. This method mayfurther comprise: (f) unlocking the contactors of the stabilizers uponcompletion of the maintenance operation; (g) retracting the contactorsof the stabilizers; (h) moving the robotic mobile platform to a secondlocation where the tool is able to reach a second area of interest onthe surface of the target object; (i) configuring the robotic mobileplatform such that the end effector is at a correct location relative tothe second area of interest; (j) extending contactors of a plurality ofstabilizers into contact with the surface of the target object; (k)locking the contactors of the stabilizers in place to maintain the endeffector at the correct location relative to the second area ofinterest; and (l) performing a second maintenance operation using thetool while the end effector is maintained at the correct locationrelative to the second area of interest.

A further aspect of the subject matter disclosed in detail below is amethod for performing a maintenance operation, comprising: (a) couplinga tool to an end effector, which end effector in turn is coupled to adistal end of an arm of an automated apparatus; (b) activating theautomated apparatus to perform an alignment operation that locates thetool at a location relative to an area of interest on a surface of atarget object; (c) extending contactors of a plurality of stabilizerscoupled to the distal end of the arm into contact with the surface ofthe target object while the tool is at the location; and (d) activatingthe tool to perform a maintenance operation while the contactors are incontact with the surface of the target object. This method furthercomprises locking the contactors of the stabilizers in place while theyare in contact with the surface of the target object before activatingthe tool.

Yet another aspect is an automated apparatus comprising: a base platformcomprising wheels; an arm movably coupled to the base platform andhaving a distal end; an end effector which is coupled to the distal endof the arm; a tool which is coupled to the end effector; and a pluralityof stabilizers coupled to the tool or to the end effector, eachstabilizer comprising a rod and a contactor disposed at a distal end ofthe rod, wherein the rod is selectively extendible or retractable andthe contactor is made of elastomeric material. In accordance with someembodiments, each stabilizer comprises a pneumatic cylinder that isoperable to extend or retract the rod and a rod lock attached to thepneumatic cylinder and configured for locking the rod to preventmovement.

Other aspects of systems and methods for stabilizing an end effectorcoupled to a distal end of an arm of an extended-reach automatedapparatus are disclosed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The features, functions and advantages discussed in the precedingsection can be achieved independently in various embodiments or may becombined in yet other embodiments. Various embodiments will behereinafter described with reference to drawings for the purpose ofillustrating the above-described and other aspects. None of the diagramsbriefly described in this section are drawn to scale.

FIG. 1 is a block diagram identifying some components of a system forthermographic imaging of a fuselage section.

FIG. 2 is a diagram representing a side view of a ground-based roboticNDI mobile platform in accordance with one embodiment.

FIG. 3 is a diagram representing a side view of a ground-based roboticNDI mobile platform in accordance with another embodiment.

FIG. 4 is a diagram representing an exploded view of some components ofthe robotic NDI mobile platform depicted in FIG. 3.

FIG. 5 is a diagram representing a perspective view of a ground-basedrobotic NDI mobile platform in the process of scanning a curvedworkpiece made of composite material. (Laser range meters mounted to therobot end effector are not shown in FIG. 5, but are shown in FIG. 6.)

FIG. 6 is a diagram representing a side view of a portion of the roboticNDI mobile platform depicted in FIG. 5, which portion includes the endeffector and three laser range meters mounted to the end effector.

FIG. 7 is a diagram representing a perspective view of an infraredthermography scanner mounted to the end effector of the robotic NDImobile platform depicted in FIG. 5.

FIG. 8 is a diagram representing a front view of the infraredthermography scanner depicted in FIG. 7.

FIG. 9 is a block diagram identifying some components of an alignmentsystem in accordance with some embodiments.

FIG. 10 is a diagram representing a scan pattern (3×2) for infraredthermographic inspection of a large workpiece.

FIG. 11 is a flowchart identifying some steps of a method ofnon-destructive inspection that employs an end effector alignmentprocess in accordance with one embodiment.

FIGS. 11A and 11B (in conjunction) form a flowchart identifying somesteps performed by the alignment process finite-state machine employedin the method depicted at a high level in FIG. 11.

FIG. 12 is a diagram representing measurement of a scan region boundaryusing a local positioning system (LPS).

FIG. 13 is a diagram representing initial location alignment of therobotic system using the LPS.

FIG. 14 is diagram representing a perspective view of the LPS executinga robot-to-part localization process in accordance with one embodiment.

FIGS. 15A through 15C are diagrams representing front, side and topviews respectively of three laser range meters arranged in a triangularpattern in a common plane and directed at respective spots on a surfaceof a target object, the laser range meters and spots being separated byrespective distances.

FIG. 16 is a diagram representing a top view of a holonomic-motion baseplatform having four Mecanum wheels with various dimensions indicated bydouble-headed arrows.

FIG. 17 is a block diagram identifying some components of a system forinfrared thermographic inspection of large composite structures inaccordance with some embodiments.

FIG. 18 is a diagram representing a perspective view of a ground-basedrobotic NDI mobile platform in the process of scanning a curvedworkpiece using a pneumatically stabilized scanner in accordance withone embodiment.

FIG. 19 is a flowchart identifying steps of a method for performing amaintenance operation using a robotic mobile platform equipped with amaintenance tool and a plurality of stabilizers.

FIG. 20 is a diagram representing a view of a stabilized hybrid toolassembly mounted to an end effector in accordance with an alternativeembodiment.

FIGS. 20A and 20B are diagrams representing magnified views ofrespective portions of the hybrid tool assembly within the respectivedashed rectangles 20A and 20B seen in FIG. 20.

FIGS. 21 and 22 are diagrams representing front and side viewsrespectively of the hybrid tool assembly depicted in FIG. 20.

FIG. 23 is a diagram identifying some components of a system forstabilizing a distal end of an extended-reach arm using pneumaticcylinders having rod lock mechanisms.

FIG. 24 is a block diagram identifying mechanical couplings used tocouple an ultrasonic transducer array and a stabilizer to a distal endof an arm of an automated apparatus in accordance with one embodiment.

FIG. 25 is a block diagram identifying components of a control systemthat uses rotation encoders to track the relative location (e.g.,relative to an initial location acquired using a local positioningsystem) of an NDI sensor unit of the type depicted in FIG. 20.

FIG. 26 is a block diagram identifying some components of a stabilizerfixedly coupled to an end effector, which stabilizer comprisestelescoped tubes, a spring and a contactor.

FIG. 27 is a diagram representing a side view of a ground-based roboticNDI mobile platform having a pneumatically stabilized distal end of anarm in accordance with another embodiment.

Reference will hereinafter be made to the drawings in which similarelements in different drawings bear the same reference numerals.

DETAILED DESCRIPTION

Many large structures, including aircraft, generally undergo routinemaintenance, including inspection and/or repair of various components.It is known to use an automated tool-equipped apparatus (e.g., a roboticmobile platform) in fabrication, repair and maintenance of largestructures. Typically, the tool is coupled to an end effector, which endeffector in turn is coupled to a distal end of an arm of the automatedapparatus. That arm may consist of a single rigid link or multiple rigidlinks rotatably coupled in sequence by joints or translatably coupledfor extension/retraction.

The efficacy of an automated maintenance operation may be diminished ifthe distance between the end effector and the proximal end of the armexceeds a specified threshold due to a loss of positional andorientational control at the distal end of the arm. More specifically,robots being developed for automated fabrication, repair, and inspectionof composite aircraft structure must extend their reach beyond currentmulti-axis pedestal-mounted robots. At the same time, their preciseactivities (such as creating a scarf repair or scanning during anon-destructive inspection) demand a positionally stable distal workingend.

At least some current solutions for stabilizing arms of automatedapparatus add weight, size, and expense to the design and limit theusefulness, portability, and applicability of the apparatus. Other knownsolutions are to use vacuum track scanning systems (e.g., a scanningsystem in which the scanner is mounted to a bridge that moves alongtracks adhered to the surface of the workpiece using vacuum adherencedevices) or crawling robots that mount or attach to the surface (e.g.,robotic crawlers having a plurality of independently controlled suctiondevices). These provide end effector stability, but require directmount/dismount activities, which activities may have safety, ergonomic,potential damage and set-up time drawbacks.

The stabilization devices disclosed in some detail below may beinstalled on any robotic apparatus having an end effector mounted to adistal end of an articulated or non-articulated arm. In the examplesdescribed below, an end effector is pivotably coupled to a rigidextension arm, which arm is not articulated. However, in alternativeembodiments the extension arm may have an additional actuatedtranslational or rotational degree-of-freedom. Any one of a multiplicityof interchangeable maintenance tools of different types can be attachedto and later detached from the end effector depending on which tool ismost suitable for performing a scheduled maintenance operation.

As used herein, the term “tool” means any device that can be mounted toan end effector at the distal end of an arm of a robotic apparatus andused in the course of executing an automated fabrication, repair ormaintenance procedure, Each maintenance tool incorporates elements toperform its intended function. Some examples of suitable maintenancetools include, but are not limited to, the following: (a) a video cameraand a source of light to enable visual inspection for damage inside thelimited-access space; (b) an NDI sensor unit (e.g., an array ofultrasonic transducers, an eddy current probe, microwave sensors, aninfrared thermography unit, etc.); (c) a laser line scanner or athree-dimensional mapping head for dimensional measurement or mappingdamage at a location; (d) other sensor or sensor array; (e) amini-sander or grinder to remove material during a repair; (f) a vacuumto pick up foreign object debris or repair debris; (g) a spray head forpainting or coating a repaired area; (h) an applicator for applying acoating; (i) a scaler; (j) a drill; and (k) other tools for maintenanceor repair.

For the purpose of illustration, systems and methods for non-destructiveinspection of fuselage sections made of composite material (e.g., acomposite laminate made of fiber-reinforced plastic) will now bedescribed in detail. More specifically, first a robotic apparatus havingan infrared thermography subassembly attached to an end effector will bedescribed. That robotic apparatus may be modified to also performultrasonic inspection by mounting an ultrasonic inspection assembly tothe housing of the infrared thermography device. In accordance with afurther embodiment, the ultrasonic inspection assembly can be mounted tothe end effector of a robotic apparatus without an infrared thermographydevice.

Infrared thermography methods and devices make it possible to performnon-destructive testing of a material to detect defects, variations inthe properties of the material, or differences in thickness of a coatingor layer of the material. Infrared imaging can detect local variationsin thermal diffusivity or thermal conductivity at or beneath the surfaceof the material. Infrared thermography can be used on metals, such asferrous materials, including steel, or on non-metallic materials, suchas plastics, ceramics, or composite materials.

Active thermography is used to nondestructively evaluate samples forsub-surface defects. It is effective for uncovering internal bonddiscontinuities, delaminations, voids, inclusions, and other structuraldefects that are not detectable by visual inspection of the sample.Generally, active thermography involves heating or cooling the sample tocreate a difference between the sample temperature and the ambienttemperature and then observing the infrared thermal signature thatemanates from the sample as its temperature returns to ambienttemperature. An infrared camera is used because it is capable ofdetecting any anomalies in the cooling behavior, which would be causedby sub-surface defects blocking the diffusion of heat from the samplesurface to the sample's interior. More particularly, these defects causethe surface immediately above the defect to cool at a different ratethan the surrounding defect-free areas. As the sample cools, theinfrared camera monitors and records an image time sequence indicatingthe surface temperature, thereby creating a record of the changes in thesurface temperature over time.

Typically, the surface of the material is heated using a flash lamp andafter a fixed period of time, a thermal image is taken of the surface ofthe heated material. Systems for thermographic heating typically employxenon flashtubes and off-the-shelf photographic power supplies forsample excitation. An infrared camera images the infrared spectralradiance from the surface of the material, which is representative ofthe temperature of the surface of the material. Differences intemperature of the surface of the material indicate differing thermalcharacteristics of the material. These variations in thermalcharacteristics of the material indicate a possible material defect orthe inclusion of a foreign material.

Structural thickness and stack-up geometry needed for infrared signatureprocessing is obtained by knowing the exact location of the infraredcamera's field of view on the surface of the fuselage section.

FIG. 1 is a block diagram identifying some components of a system forthermographic imaging of a fuselage section 2. This infraredthermographic inspection system comprises a digital infrared camera 4having a lens that is directed through a camera lens aperture 5 in ahood 12, which is designed to form an enclosure adjacent to the surfacebeing inspected. A pair of flash lamps 6 a and 6 b are disposed insideand in fixed spatial relationship to the hood 12. The flash lamps 6 aand 6 b produce flashes of light in response to trigger signals from aninfrared thermography processing control computer (hereinafter “infraredthermography computer 8”), which also controls operation of the infraredcamera 4. One example of a type of infrared camera 4 suitable for usewith at least some of the embodiments disclosed herein includes a focalplane array (FPA) device configured to act as a spectral radiometer.Further details concerning other components that may be included in aflash lamp assembly of a type comprising an infrared camera, a pair offlash lamps and a hood can be found, for example, in U.S. Pat. No.7,186,981.

In accordance with one method of thermographic inspection, first theflash lamps 6 a and 6 b are triggered to transfer heat to the compositematerial of the fuselage section 2. Preferably, during cooling of thecomposite material, the infrared camera 4 is triggered periodically tocapture successive digital images of the varying spectral radiance ofthe heated portion of the fuselage section 2. Preferably, the thermallyexcited (heated) region of the composite material being inspected willcool monotonically after the excitation source removed until the samplereaches thermal equilibrium with its surroundings. The digital infraredimaging data captured by infrared camera 4 is received by the infraredthermography computer 8 for processing. The infrared thermographycomputer 8 is programmed to process infrared imaging data to detect andlocate material edges, foreign objects under the surface of thematerial, or other material anomalies, such as delaminations andout-of-tolerance porosity. The infrared imaging data may be displayed ona display monitor (not shown in FIG. 1), which may be integrated with orseparate from infrared thermography computer 8.

In accordance with the embodiment depicted in FIG. 1, the infraredthermography computer 8 may have digital image acquisition capabilitiesto convert the infrared imaging data from the infrared camera 4 to aformat that can be analyzed and mathematically manipulated by theinfrared thermography computer 8. An optional data acquisition module 10may be incorporated in or separate from (as depicted in FIG. 1) theinfrared thermography computer 8. The data acquisition module 10 may beused if the infrared camera 4 captures multiple spatially differentimages to generate a complete mosaic image of the surface of thecomposite structure when the latter is too large to fit in a singleimage frame. The infrared thermography computer 8 may be furtherprogrammed to analyze the infrared imaging data captured by the infraredcamera 4. In particular, the time history of the surface temperatureresponse of the fuselage section 2 as it returns to room temperature canbe analyzed to detect the presence of defects in the composite material.

In the context of the specific application of inspecting fuselagesections, a non-destructive inspection system may comprise means forscanning the skin of the fuselage section from a vantage point externalto the fuselage section. In the embodiments disclosed below, theexternal scanning means comprise a robot equipped with an infraredcamera. The robot comprises a movable robot base and a robotic armhaving a proximal end coupled to the robot base. The robot base may be amobile holonomic crawler vehicle. An infrared thermography scanner iscoupled to a distal end of the robotic arm. The infrared thermographyscanner comprises an infrared camera and two or more flash lampsattached inside a hood. The hood may be sized to cover a rectangulararea on the outer surface of the fuselage section. The infrared imagingdata acquired from adjacent rectangular areas can be stitched togetherbased on measurements of the respective locations of the robot baseusing a local positioning system. The stitching process may be performedon a real-time basis or may be performed at a later time.

Various embodiments of NDI systems configured to use the locationalignment feedback concepts disclosed herein will now be described insome detail. In accordance with some embodiments, the NDI system is anautomated platform with an end effector that is able to reach to thecenterline of the top and bottom of the fuselage from either side of theairplane. This NDI system comprises a Mecanum-wheeled holonomic-motionbase platform, a vertical extension mast carried by the base platform, apivoting end effector, proximity sensors, and support for multiple typesof NDI devices mounted on the end effector. The vertical support mastwith a pivoting end effector on an extension arm allows the inspectionof the full height of an airplane fuselage section. The holonomic-motionbase platform allows the robot to quickly and efficiently re-positionthe NDI scanner unit along the length of the fuselage. Motion controlsoftware with distance sensor feedback enables automatic captureoverlapping grid pattern scans. Reference position data is also capturedto align the NDI scans with the appropriate airplane coordinate system.The system is relative easy to set up and use in either the automated ormanual control mode. It can be configured to accept various types of NDIunits mounted to its end effector, including: eddy current, ultrasonic,and infrared thermography (IRT) NDI sensors.

FIG. 2 is a diagram representing a side view of a ground-based roboticNDI mobile platform 200 in accordance with one embodiment. This platformcomprises a holonomic-motion base platform 204, an infrared thermography(IRT) scanner 214, and an automated scanner support apparatus (carriedby the holonomic-motion base platform 204) that is under the control ofa robot controller (not shown). The automated scanner support apparatuscomprises a vertical extendible mast 206 that can be extended andretracted as needed to change the elevation of the IRT scanner 214. Thevertical extendible mast 206 comprises a first mast section 206 a havinga linear axis and one end fixedly coupled to a holonomic-motion baseplatform 204, a second mast section 206 b having a linear axis and whichis slidably coupled to the first mast section 206 a for sliding along aline that is parallel to the axis of the first mast section 206 a, and athird mast section 206 c having a linear axis and which is slidablycoupled to the second mast section 206 b for sliding along a line thatis parallel to the axis of the second mast section 206 b. In accordancewith one implementation, the vertical extension of the mast iscontrolled by a single motor and a cable-pulley system.

The ground-based robotic NDI mobile platform 200 depicted in FIG. 2further comprises a four-bar linkage arm mechanism 208 to control theposition and orientation of an end effector 212 which is pivotablycoupled to the distal end of the four-bar linkage mechanism. The driverlink of the four-bar linkage mechanism 208 is driven to rotate relativeto the third mast section 206 c by a motor-driven lead screw orhydraulic cylinder 210. The IRT scanner 214 is mounted to and rotateswith the end effector 212. An IRT shroud 216 surrounds the IRT scanner214 to isolate the volume of space between the IRT scanner 214 and acurved workpiece 202 (e.g., a fuselage section) from the surroundingenvironment.

FIG. 3 is a diagram representing a side view of a ground-based roboticNDI mobile platform 220 in accordance with another embodiment. Thisembodiment comprises a vertical extendible mast 206, a rigid extensionarm 222 fixedly coupled to the third mast section 206 c, and an endeffector 224 pivotably coupled to the distal end of the two sides of therigid extension arm 222. FIG. 4 is a diagram representing an explodedview of some components of the robotic NDI mobile platform 220 depictedin FIG. 3. This embodiment has independent programmable control over theextension height as well as the end effector pitch. Pitch rotation ofthe end effector 224 can be driven by a position control motor 246 (seeFIG. 4) with a non-back-drivable gearbox (not shown).

FIG. 5 is a diagram representing a perspective view of a prototype of aground-based robotic NDI mobile platform 230 in the process of scanninga curved workpiece 202 made of composite material. The IRT scanner 214is mounted to an end effector 224, which end effector 224 in turn ispivotable about a pitch axis under the control of a robot controller 80.The end effector 224 is pivotably coupled to a rigid extension arm 232which is in turn fixedly coupled to the uppermost mast section of thevertical extendible mast 206. The IRT scanner 214 sends acquired data toan infrared thermography computer (not shown in FIG. 5) by way of anelectrical cable 242. The robotic NDI mobile platform 230 is alsoequipped with a warning light 244 which switches on and off when thesystem is enabled.

In accordance with one proposed implementation, the holonomic-motionbase platform 204 employs four Mecanum wheels arranged with a Type Apair on one diagonal and a Type B pair on the other. The Type A Mecanumwheels differ from the Type B Mecanum wheels in that the tapered rollersof the former are oriented at different angles than the tapered rollersof the latter. Each Mecanum wheel can be driven to rotate by arespective independently controlled stepper motor. A Mecanum-wheeledvehicle can be made to move in any direction and turn by controlling thespeed and direction of rotation of each wheel. For example, rotating allfour wheels in the same direction at the same rate causes forward orbackward movement; rotating the wheels on one side at the same rate butin the opposite direction of the rotation by the wheels on the otherside causes the vehicle to rotate; and rotating the Type A wheels at thesame rate but in the opposite direction of the rotation of the Type Bwheels causes sideways movement. The holonomic-motion base platform 204moves under the control of an onboard control computer (i.e., robotcontroller). The structure and function of a suitable Mecanum-wheeledholonomic-motion base platform is described in U.S. Pat. No. 9,410,659,the disclosure of which is incorporated by reference herein in itsentirety.

In accordance with one embodiment, a multiplicity of sensors (not shownFIG. 5) are mounted around the periphery of the holonomic-motion baseplatform 204 to indicate the presence of obstacles in that specificregion of the vehicle. The motion controller uses that sensor data toblock additional motion from occurring in the direction associated withthat particular sensor, but motion in the other directions is stillallowed. Potential sensors include contact sensors, thru-beam sensors,and proximity sensors. This collision avoidance system would operate ina similar manner similar to what is described in U.S. Pat. No.7,194,358.

As previously mentioned, the location alignment feedback processdisclosed herein employs distance sensors to determine the position andorientation (i.e., the location) of the IRT scanner 214 relative to thetarget object (e.g., workpiece 202). At least three non-collineardistance measurement devices can be used to compute relative location inreal-time. To mitigate any possibility of scratching the surface of thetarget object, laser range meters were selected instead of contactprobes for use as distance sensors. In addition to close-range distanceand angle guidance, the laser range meters also provide the advantage oflonger range distance feedback to the platform motion controller forgeneral navigation purposes.

In accordance with one implementation, three laser range meters (notshown in FIG. 5) are mounted to the end effector 224. FIG. 6 is adiagram representing a side view of a portion of the robotic NDI mobileplatform depicted in FIG. 5, which portion includes the end effector 224and three laser range meters mounted to the end effector 224. Only twoof the three laser range meters (i.e., laser range meters 236 and 238)are visible in FIG. 6. The third laser range meter (i.e., laser rangemeter 240) is visible in FIG. 7. As seen in FIG. 6, the first laserrange meter 236 is attached to an L-shaped mounting plate 218 a, whichis in turn attached to the end effector 224. Similarly, the second laserrange meter 238 is attached to an L-shaped mounting plate 218 b (shownin FIGS. 6 and 7), which is in turn attached to the end effector 224;and the third laser range meter 240 is attached to an L-shaped mountingplate 218 c (shown in FIG. 7), which is in turn attached to the endeffector 224.

FIG. 7 is a diagram representing a perspective view of the IRT scanner214 (with shroud 216 removed) mounted to the end effector 224, which inturn is pivotably coupled to the rigid extension arm 232. As previouslydescribed, laser range meters 236, 238 and 240 are mounted to the endeffector 224. As best seen in the front view of FIG. 8, laser rangemeter 236 is mounted at an elevation which is higher than the elevationof the highest point of the hood 12 of the IRT scanner 214, whereas thelaser range meters 238 and 240 are mounted at an elevation which islower than the elevation of the lowest point of the hood 12 of the IRTscanner 214, separated by a distance. Preferably the laser range meters236, 238 and 240 are disposed at the vertices of an isosceles triangle.FIG. 15A shows an arrangement wherein the distance separating the laserrange meters 238 and 240 (i.e., the base of the isosceles triangle) is aand the distance separating the laser range meter 236 and a point midwaybetween the laser range meters 238 and 240 (i.e., the height of theisosceles triangle) is b.

The system depicted in FIGS. 5-7 uses an on-board alignment system todetermine relative location (position and orientation) offsets of theend effector 224 to a target object. This process uses distanceinformation from the laser range meters 236, 238 and 240 to computerelative location in real-time. The system then provides that data to arobot controller to produce the desired feedback-based motion of the endeffector 224 (which may also include motion control of other parts ofthe robot).

One form of control that this process enables is semi-automated controlto assist an operator in some aspect of alignment, such as orientationof the end effector 224 to make sure that it is always perpendicular tothe surface of the target object or making sure that it is always aspecific distance from the surface.

FIG. 9 is a block diagram identifying some components of an alignmentsystem in accordance with some embodiments. The distance sensors 14(e.g., laser range meters 236, 238 and 240) provide distance informationto a computer 16 (e.g., a robot controller). The computer 16 isconfigured (e.g., programmed) to determine movements that will align theend effector 224 with the surface of the target object based on thedistance information received from the distance sensors 14. Thesemovements may include one or more of the following: moving theholonomic-motion base platform 204 to a new location; extending orretracting the vertical extendible mast 206; and pivoting the endeffector 224 about the pitch axis. The robotic NDI mobile platformcomprises a multiplicity of motors 20 which are controlled by respectivemotor controllers 18. The computer 16 sends command signals to selectedmotor controllers 18 to activate the robotic movements to align the endeffector 224 with the surface of the target object.

Another form of control enabled by this process is fully automatedmotion control, where the operator specifies a high-level goal, such asan m×n grid pattern, and then the automated controller does the motionplanning based on the high-level goal and feedback from the alignmentsystem. For example, FIG. 10 is a diagram representing a 3×2 scanpattern 22 for IRT inspection of a large workpiece. First the IRTscanner acquires IRT data for scan area 26 a. Then the IRT scanner ismoved upward and stopped at a location where it acquires IRT data forscan area 26 b. Preferably scan area 26 b overlaps scan area 26 aslightly to facilitate stitching the scans together and ensure thatthere are no gaps in the coverage. Next the IRT scanner is movedrightward and stopped at a location where it acquires IRT data for scanarea 26 c. Then the IRT scanner is moved downward and stopped at alocation where it acquires IRT data for scan area 26 d, followed by amove rightward to acquire IRT data for scan area 26 e, and then a moveupward to acquire IRT data for scan area 26 f. The scan path 28 of theIRT scanner during this process is indicated by arrows in FIG. 10.

The alignment process provides an alternative to directly programmingthe individual motions of the robot. It also enables the system to adaptto unexpected changes in the environment, as well as providing collisionavoidance ability for the end effector to achieve the desired positionand orientation relative to the target surface without contacting it.

The automated process used here is based on a finite-state machinecontrol application that manages the transition from one state toanother based on external inputs. This framework enables the system toproduce responses based on multiple types of inputs and the currentstate of the system. The various actions of the system used to producethe automatically generated motion path plan and scanner control signalsare based on satisfying the criteria for transitions between one mode ofoperation and another. In accordance with one embodiment, a finite-statemachine uses sensor feedback to trigger transitions between discretesets of system states.

The process will now be described with reference to a robotic NDI mobileplatform comprising a base platform (e.g., a holonomic-motion baseplatform) carrying a robotic arm having an NDI sensor (e.g., an IRTscanner) at its distal end, the movements of the base platform and therobotic arm being controlled by a device controller (e.g., a robotcontroller). FIG. 11 shows the high-level process for the overalloperation of the system, and FIGS. 11A and 11B show the detailsassociated with the alignment-bases aspects of the process. Digitalsignals sent between the robot controller and the NDI sensor controlcomputer (e.g., infrared thermography computer 8 identified in FIG. 1)enable synchronization between the separate robot and NDI sensorsystems.

FIG. 11 identifies some steps of a method 100 of non-destructiveinspection that employs an end effector alignment process in accordancewith one embodiment. To start the process, the system operatoridentifies a first location for the NDI sensor relative to the targetobject (step 102). This step can be accomplished by visual methods (bythe operator), or automatically (with a pointing system like the LPS).Then the system operator can operate the base platform and robotic armto move the NDI sensor to the approximate first location (step 104). Thesystem operator provides the desired number of scans in the pattern tothe device controller in step 106. This number will be compared to acount stored in a grid location counter, which counter is incrementedeach time a scan in the pattern is acquired. In step 108, the devicecontroller computes preliminary path parameters and begins automatedscan acquisition. The system operator also enables proximity/collisiondetection (step 110). Then the system operator captures the 3-Dcoordinates of the first location using an external tracking system(e.g., an LPS) (step 112). (The external tracking system has beenpreviously calibrated so that its own 3-D coordinates relative to thecoordinate system of the target object are known, which allows the LPScomputer to compute the 3-D coordinates of the first location relativeto the coordinate system of the target object.) Thereafter afinite-state machine for controlling the motion of the NDI sensor duringthe alignment process and during the scanning process is enabled (step114) (i.e., go to A in FIG. 11A). (The finite-state machine will bedescribed in the next paragraph with reference to FIGS. 11A and 11B.)After the NDI sensor has been aligned and the scan pattern has beencompleted, the system operator captures the 3-D coordinates of theending location of the NDI sensor using the external tracking system(step 116). A combined image can then be assembled by stitching togetherthe scan data from adjoining scans (step 118).

FIGS. 11A and 11B (in conjunction) form a flowchart identifying somesteps performed by the finite-state machine employed in the methoddepicted at a high level in FIG. 11. A finite-state machine is amathematical model of a process that can only be in one of a finitenumber of states at any given time.

In accordance with one proposed implementation, the robot controllerfirst checks (i.e., determines) whether the finite-state machine (FSM)is set to the GRID_MOVE state or not (step 120). GRID_MOVE is the statewhere the robot is moving between the grid locations which are definedat a high level. For example, if the system operator wants the system tocapture data in a 3×2 pattern, the robot will move along the scan path28 seen in FIG. 10 to make a contiguous grid. If the robot controllerdetermines in step 120 that the FSM is not in the GRID_MOVE state, therobot controller then proceeds directly to step 128. If the robotcontroller determines in step 120 that the FSM is in the GRID-MOVEstate, the robot controller then determines whether there are additionalgrid locations in the sequence (step 122). This is accomplished bycomparing the current count in the grid location counter to the presetnumber of scans to be acquired. If the robot controller determines instep 122 that there are no additional grid locations (i.e., the countequals the present number) in the sequence, the process returns to step116 in FIG. 11. If the robot controller determines in step 122 thatthere are additional grid locations in the sequence (i.e., the count isless than the preset number), the robot moves to the next location ofthe unaligned NDI sensor (step 124), following which the state of thefinite-state machine is set to ALIGN (step 126). In the next step, therobot controller determines whether the finite-state machine is set tothe ALIGN state or not (step 128).

The ALIGN state is when the robot is using the three distance sensors toensure that the pitch and yaw of the end effector are such that the aimaxis of the NDI scanner is perpendicular to the surface of the targetobject. If the robot controller determines in step 128 that thefinite-state machine is not in the ALIGN state, then the robotcontroller proceeds directly to step 144 in FIG. 11B. If the robotcontroller determines in step 128 that the finite-state machine is inthe ALIGN state, then the robot controller determines whether thelocation of the NDI sensor needs to be refined or not (step 130). If therobot controller determines in step 130 that the location of the NDIsensor does not need to be refined (i.e., the aim axis of the NDIscanner is perpendicular to the surface of the target object), then therobot controller sets the state of the finite-state machine to SCAN(step 132) and proceeds directly to step 144 in FIG. 11B. If the robotcontroller determines in step 130 that the location of the NDI sensordoes need to be refined (i.e., the aim axis of the NDI scanner is notperpendicular to the surface of the target object), then the robotcontroller performs the following steps in order: (a) acquires distancedata from the distance sensors (step 134); (b) computes the orientationand translation offsets from the desired aligned location (step 136);(c) align the distance to the desired offset (step 138); (d) aligns theyaw angle of the end effector to achieve perpendicularity to the surfaceof the target object and adjust lateral position (step 140); (e) alignsthe pitch angle of the end effector to achieve perpendicularity to thesurface of the target object and adjust height (step 142); and (f)returns to step 130.

As previously described, if the robot controller determines in step 130that the location of the NDI sensor does not need to be refined, thenthe robot controller sets the state of the finite-state machine to SCAN(step 132) and proceeds directly to step 144 in FIG. 11B. In step 144,the robot controller determines whether the finite-state machine is setto the ALIGN state or not. If the robot controller determines in step144 that the finite-state machine is not in the SCAN state, then therobot controller returns to step 120 in FIG. 11A. If the robotcontroller determines in step 144 that the finite-state machine is inthe SCAN state, then the robot controller sends scanner control commands(step 146) to the NDI scanner control computer (e.g., infraredthermography computer 8 identified in FIG. 1). The robot controller thenchecks for a scanner response (step 148) and determines whether the scanpattern has been completed or not (step 150). If the robot controllerdetermines in step 150 that the scan pattern has not been completed,then the robot controller returns to step 148. If the robot controllerdetermines in step 150 that the scan pattern has been completed, thenthe robot controller performs the following steps in order: (a) returnsthe NDI scanner to the location (step 152); (b) sets the state of thefinite-state machine to GRID_MOVE; (c) increments the grid locationcounter; and (d) returns to step 114 in FIG. 11.

After the automated scanning sequence is complete, the individual imagesfrom each IRT scan can then be stitched together to produce a singlerepresentation of the inspection region.

The above-described system may have many potential use cases for generalalignment tasks of robotic manipulators or other devices. One of theseuse cases is for grid-based NDI scan acquisition in aerospacemanufacturing and maintenance environments, e.g., grid-based scanning ofan airplane fuselage.

During typical operation, this system can be driven (teleoperated) by auser to an approximate first location, after which it is set toautomatically acquire grid scans arranged in an operator-definedvertical and horizontal pattern along either side of the airplanefuselage, as shown in FIG. 10.

The automated grid scanning feature of the motion control algorithminvolves feedback of distance data from three laser range meters 236,238 and 240 to the motion control algorithm, which sets the horizontaland vertical placement, as well as the yaw and pitch orientation of theplatform 204 and end effector 224, respectively. This approacheliminates the need for individual pre-defined motion paths for thesystem, which simplifies usage and reduces setup time.

The system can also be fully controlled in a teleoperation mode to allowoperators to acquire data manually. A semiautomated mode is alsopossible, where the system operator controls the platform location andmast height, and the system automatically adapts the end effector pitchorientation to maintain a perpendicular alignment with the surface infront of it.

In order to correctly locate the scans in the coordinate system of anairplane, 3-D coordinate position measurements are taken of the boundaryregions of the scans. This boundary reference allows the combined scanimage to be placed in the same coordinate system as the target objectand its associated CAD models. This enables association of acquiredscans with the respective 3-D models of the target object, as well asproviding location data for future reference. For this system, a localpositioning system (LPS) 24 (shown in FIG. 12) is used to acquire 3-Dcoordinate position data in the coordinate system of the target object54. For example, FIG. 12 shows the LPS 24 directing a laser beam 30 at ameasured boundary position 60. Assuming that the LPS 24 has already beencalibrated with respect to the coordinate system of the target object54, the boundary position data points acquired by the LPS 24 can be usedto determine the coordinates of each boundary position in the coordinatesystem of the target object 54.

In accordance with one embodiment, acquisition of the boundaries of ascan can be accomplished by targeting the corners of the IRT shroud 216when the IRT scanner 214 (see FIGS. 5 and 8) is capturing the scan datafor specific locations. These LPS boundary measurements are performedbefore the first scan and after the last scan, or at any intermediatelocation in the grid sequence. In accordance with one proposedimplementation, the corners (or some known locations) on the IRT shroud216 can have either active (e.g., LEDs) or passive optical targets orother visible features. The passive approach requires that the systemoperator run the LPS 24 to target the points; the use of active LEDtargets enables an automated approach using the LPS camera to detect theLEDs. Ideally it would be best to get all four corners of the scannedregion, but the IRT shroud 216 sometimes occludes the optical targets,which makes them hard to target. The minimum number of optical targetsneeded for this part of the process is two, since one can makeassumptions about the shape of the X by Y scanned region, for example,using surface normals from a 3-D CAD model of the target object.

The motorized pan-tilt control aspects of the LPS 24 allow it to alsoprovide an initial location reference and guidance capability toindicate a desired first location for scans. After an initialcalibration of the LPS 24 to known locations on the target object, theoperator can instruct the LPS 24 to aim its laser pointer at a specified3-D coordinate on the target surface, which laser spot 38 is indicatedin FIG. 13. The operator then drives the robot to align the laser spots32 a, 32 b and 32 c of its laser range meters 236, 238 and 240respectively around the LPS laser spot 38, as shown in FIG. 13.

In order to use the LPS 24 to acquire measurements in the coordinatesystem of the target object 54 (e.g., an airplane), the system operatoruses three known points on the target object. These three points arecalibration points, which are separate from the points on the IRT shroud216 that are measured for the scan registration process. This means thatif one wants to align the scan data with the airplane coordinates, thetotal minimum number of LPS measurements is five: three for the initialLPS calibration and two to define the rectangular region in which thescan took place.

FIG. 14 is a perspective view of a system capable of executing arobot-to-target localization process in accordance with one embodiment.The robot-to-target localization process is performed using an LPS 24comprising a single camera 40 and a laser range meter (not shown) on acontrollable pan-tilt unit 42. The LPS operation and calibration processis disclosed in U.S. Pat. No. 7,859,655, the disclosure of which isincorporated by reference herein in its entirety.

More specifically, the local positioning system depicted in FIG. 14comprises a video camera 40 which may have automated (remotelycontrolled) zoom capabilities. The video camera 40 is supported on apan-tilt mechanism 42. The video camera 40 and the pan-tilt mechanism 42may be operated by an LPS control computer 48. The LPS control computer48 communicates with the video camera 40 and the pan-tilt mechanism 42through a video/control cable 46. Alternatively, the LPS controlcomputer 48 may communicate with video camera 40 and pan-tilt mechanism42 through a wireless communication pathway (not shown). The LPS controlcomputer 48 is configured to control the operation of the LPS hardware,including the laser range meter (not shown), the video camera 40 and thepan-tilt mechanism 42. For example, the pan and tilt angles of thepan-tilt mechanism 42 and, therefore, the orientation of the videocamera 40 can be controlled using the keyboard of computer 48 or otheruser interface hardware 36 (e.g., a gamepad). The optical image field,as sighted by the video camera 40, can be displayed on the monitor 33 ofLPS control computer 48.

The pan-tilt mechanism 42 is controlled to rotationally adjust the laserrange meter (not shown) and the video camera 40 to selected anglesaround a vertical, azimuth (pan) axis and a horizontal, elevation (tilt)axis. A direction vector 66 (indicated by a dashed line in FIG. 14),that describes the orientation of the laser range meter (not shown) andvideo camera 40 relative to the fixed coordinate system of the tripod 44(or other platform on which the pan-tilt unit is attached), isdetermined from the pan and tilt angles when the camera is aimed at apoint of interest. In FIG. 14, the direction vector 66 extends from thelaser range meter (not shown) and video camera 40 and intersects a point94 a on one corner of the shroud 216.

A laser range meter may be incorporated inside the housing of camera 40or mounted to the outside of camera 40 in such a way that it transmits alaser beam along the direction vector 66. The laser range meter isconfigured to measure the distance to any visible feature (e.g., one ofthe corners 94 a-c) on the shroud 216 or the distance to any calibrationpoint (e.g., points 92 a-c) on the curved workpiece 202. (Eachcalibration point may be a visible feature on the curved workpiece 202or an optical target attached to the curved workpiece 202.) The laserrange meter may have a laser and a unit configured to compute distancesbased on the laser light detected in response to a laser beam reflectedfrom the point of impingement.

The local positioning system shown in FIG. 14 further comprisesthree-dimensional localization software which is loaded into LPS controlcomputer 48. For example, the three-dimensional localization softwaremay be of a type that uses multiple calibration points 92 a-c on thecurved workpiece 202 to define the location (position and orientation)of video camera 40 relative to curved workpiece 202. The calibrationpoints 92 a-c may be visible features of known position in the localcoordinate system of the curved workpiece 202 as determined from athree-dimensional database of feature positions (e.g., a CAD model) orother measurement technique. During the LPS calibration process, X, Y, Zdata for at least three non-collinear points are extracted from the CADmodel. Typically calibration points are selected which correspond tofeatures that can be easily located on the target object. Thethree-dimensional localization software utilizes the X, Y, Z data of thecalibration points 92 a-c and the pan and tilt data from the pan-tiltmechanism 42 to define the relative position and orientation of thevideo camera 40 with respect to the local coordinate system of thecurved workpiece 202. The measured distances to the calibration points92 a-c may be used in coordination with the pan and tilt angles from thepan-tilt mechanism 42 to solve for the camera position and orientationrelative to the curved workpiece 202. A method for generating aninstrument-to-target calibration transformation matrix (sometimesreferred to as the camera pose) is disclosed in U.S. Pat. No. 7,859,655.Using the known and measured data, the calibration process computes the4×4 homogeneous transformation matrix that defines the position andorientation of the video camera 40 relative to the curved workpiece 202.

Once the position and orientation of the video camera 40 with respect tothe curved workpiece 202 have been determined and a camera posetransformation matrix has been generated, camera pan data (angle ofrotation of the video camera 40 about the azimuth axis) and tilt data(angle of rotation of the video camera 40 about the elevation axis) maybe used in conjunction with the calculated position and orientation ofthe video camera 40 to determine the X, Y and Z coordinates of any pointof interest on the shroud 216 in the coordinate system of the curvedworkpiece 202. By locating the shroud 216 at the beginning and at theend of the scan pattern, the location of the scan pattern in thecoordinate system of the curved workpiece 202 can be determined.

More specifically, a relative localization process can be used todetermine the location of a visible feature of the shroud 216 (e.g., anyone of the corners 94 a-c depicted in FIG. 14) in the coordinate systemof the curved workpiece 202 at the beginning and end of the scanpattern. The basic process sequence as applied to the shroud 216 is asfollows: (1) The local positioning system calibrates to the coordinatesystem of the target object being inspected (e.g., curved workpiece 202)by measuring three known points 92 a-c on the target object. (2) Thelocal positioning system measures the location of a visible feature(e.g., corner 94 a) on the shroud 216 when the robot is at the beginningof the scan pattern (e.g., for scan area 26 a seen in FIG. 12). (3)Later the local positioning system is used to measure the location ofthe same visible feature or a different visible feature (e.g., corner 94b or 94 c) on the shroud 216 when the robot is at the end of the scanpattern (e.g., for scan area 26 f seen in FIG. 12). (4) This allows theoperator to determine the boundaries of the scans making up the mosaicpattern.

The LPS control software running in computer 48 computes the location ofeach visible feature on the shroud 216 relative to the coordinate systemof the curved workpiece 202. The LPS control computer 48 (see FIG. 14)sends the location data to the expert workstation 74 depicted in FIG.17, which is configured to record the location coordinates for futurereference. This location data may also be used to align the scan datawith a CAD model of the target object.

The LPS control software on computer 48 outputs the point data as X, Yand Z values, but control applications use more than just X, Y and Zdata points to provide the position and orientation of the curvedworkpiece 202. To solve the position and orientation problem, the X, Yand Z data from the three measured points 92 a-c and the knowndimensions of these points are used to compute the full6-degrees-of-freedom position and orientation representation. This iswhat the previously described localization software does. The positionand orientation format that the localization software uses is a 4×4transformation matrix, but there are other ways to represent the data.

If the system operator wishes to perform a relative LPS scan (asdescribed in U.S. Patent Application Publ. No. 2015/0268033), then theoperator can use any three non-collinear points on the target object,but does not need to know the 3-D coordinates for those pointsbeforehand (as one does with the standard LPS method). The systemoperator will not obtain the results in target object coordinates withthe relative mode, but for some applications that is not needed. Therelative LPS localization process can be used to ensure that the NDIsensor is aligned in the same region as was the situation earlier. It isalso useful for piecing together several separate scans or if it wasnecessary to move the LPS.

As previously disclosed, the system uses distance measurement devices,such as lasers, string encoders, ultrasonic sensors, with the basicrequirement being at least three non-collinear distance measurementdevices. One distance sensor configuration (described above) uses threedistance measurement lasers arranged in triangular formation. In analternative embodiment, four distance measurement lasers are arranged inrectangular formation. Regardless of which sensor configuration is used,the distance data is fed to the robot controller 80, along with endeffector orientation data. Feedback control methods can be used to driveto zero the error between the current angle and the desired angle.

A method for determining angles using laser range meters will now bedescribed with reference to FIGS. 15A-15C. FIGS. 15A through 15C arediagrams representing front, side and top views respectively of threelaser range meters 236, 238 and 240 arranged in a triangular pattern ina common plane and directed at respective spots on a surface of a targetobject 54, the laser range meters and spots being separated byrespective distances.

In addition to using the three lasers to determine distance to thetarget, they are also used to determine the yaw and pitch angles. FIG.15A shows positions of the laser range meters 236, 238 and 240 relativeto each other using a horizontal dimension a and a vertical dimension b,along with measured distances d₁, d₂, and d₃ to the surface of thetarget object 54 Equations (1) and (2) can be used to calculate thepitch and yaw angles:

PitchAngle=a tan 2(d ₁−(d ₂ +d ₃)/2,b)  (1)

YawAngle=a tan 2(d ₂ −d ₃ ,a)  (2)

where PitchAngle and YawAngle are the current computed angles for thealignment apparatus shown in FIGS. 15A-15C relative to the surface ofthe target object 54. The goal for these angles, which are measuredrelative to the surface normal at the current location, is to be equalto zero; and the process to achieve the goal angles is described below.

With the current yaw and pitch angles calculated, the system motioncontroller can use a velocity control method for the controlled motions:pan, tilt, and distance. A feedback controller, such as aproportional-integral-derivative (PID) controller, can be used to driveto zero the error between the current angle and the desired angle.Equations (3) and (4) can be used to compute the pitch and yaw motioncontrol:

PitchRate=Kp _(pitch)*(PitchAngle−PitchAngle_(goal))  (3)

YawRate=Kp _(yaw)*(YawAngle−YawAngle_(goal))  (4)

where PitchRate and YawRate describe the angular rotation rates aboutthe pitch axis of the alignment apparatus and yaw axis of the base,respectively; Kp_(pitch) and Kp_(yaw) are the proportional feedbackgains associated with the pitch and yaw axes, respectively; PitchAngleand YawAngle are the angles computed from Eqs. (1) and (2),respectively; and PitchAngle_(goal) and YawAngle_(goal) are the desiredgoal angles to which the controller is driving the system toward (asmentioned earlier, these are both zero for this example). Integral andderivative feedback may also be used, but are not shown here.

The base velocity equations are as follows:

Vel_(x) =Kp _(x)*(MinDist_(x)−offset_(x))  (5)

Vel_(y) =Kp _(y)*(MinDist_(y)−offset_(y))  (6)

where Vel_(x) and Vel_(y) are the lateral velocities of the base; Kp_(x)and Kp_(y) are the proportional feedback gains for the X and Ydirections of the base, respectively; MinDist_(x) and MinDist_(y) arethe smallest values measured by the lasers in the X and Y directions,respectively; and offset_(x) and offset_(y) are the goal offsetdistances. For some applications, the lasers are not configured tomeasure in both X and Y directions; in those cases the X or Y velocitycontrol equations associated with the alignment process would not beused.

For a holonomic-motion base platform that comprises a base frame 62, onepair of Type A Mecanum wheels W1 and W3 along one diagonal and one pairof Type B Mecanum wheels W2 and W4 along the other diagonal, kinematicscan be used to compute the four individual wheel velocities. Thedimensions of the vehicle (L and D) and desired rotation point(described by the distances a₁, a₂, b₁, b₂) are shown in FIG. 16. Theindividual wheel velocities for wheels W1 through W4 are shown in Eqs.(7) through (10):

V _(W1)=Vel_(y)−Vel_(x)+YawRate*(a ₁ +b ₁)  (7)

V _(W2)=Vel_(y)+Vel_(x)−YawRate*(a ₁ +b ₂)  (8)

V _(W3) =Vel _(y) −Vel _(x)−YawRate*(a ₂ +b ₂)  (9)

V _(W4) =Vel _(y) +Vel _(x)+YawRate*(a ₂ +b ₁)  (10)

where V_(Wi) (for i=1, 2, 3, 4) are individual wheel velocities; Vel_(x)and Vel_(y) are the lateral velocities from Eqs. (5) and (6); YawRate isthe yaw rotation rate from Eq. (4); and a₁, a₂, b₁, b₂ are the rotationpoint distances as shown in FIG. 16.

The primary pivot configurations for the end effector are the following:(a) one-axis pivot: one motor, one angle sensor; and (b) two-axisgimbal: two motors, two angle sensors.

The above-described alignment process addresses both discrete andcontinuous sensor update use cases, and the concept can also be packedas a stand-alone system or part of an existing system.

Although the concepts disclosed herein have application forholonomic-motion base platforms, variations are also applicable to othersystems. Potential use cases include: holonomic and non-holonomicplatforms; articulated robotic arms; gantry arms; hybrid motion-base/armsystems; helicopters and UAVs; cameras; lights; and tools.

The laser-based alignment process disclosed herein enables the system tobe operated without having to teach the robot on-line or preprogram itoff-line, which makes this approach easier to use. This guides the endeffector into place while adapting to unexpected changes in theenvironment. Instead of playing through a list or pre-programmed motionsteps, the system operates as a finite-state machine using feedback fromsensors to transition between the various steps in the alignment,grid-based motion, and scanning process.

The alignment sensors also provide collision avoidance ability for theend effector. The configuration of the system allows it to reach areason the fuselage all the way up to the crown (top) from a ground-basedholonomic platform. The solution provides an optional process forcollecting location reference data using an external measurement system(LPS).

The ability to collect location data defined in the coordinate system ofthe target object (e.g., an airplane) allows accurate registration ofthe scan data with CAD data for maintenance/repair use, as well as a wayto record location information for archival purposes.

The configuration of the system using a vertical extension arm with arotating wrist and modular tool mount, in addition to the alignmentsensor elements, provides a compact and relatively low-cost platformthat can reach required areas around the fuselage with a minimal groundfootprint.

The system disclosed herein can be configured to accept various types ofNDI devices mounted to its end effector, including: eddy current,ultrasonic, and infrared thermography (IRT) sensors. A vertical supportmast with a pivoting end effector on an extension arm allows full heightinspection of an airplane fuselage section. The holonomic-motion baseallows efficient re-positioning of the sensor unit along the length ofthe fuselage. Motion control software enables automatic captureoverlapping grid pattern scans. Reference position data is captured forscan alignment with airplane coordinates.

During operation, this system can be driven (teleoperated) by anoperator to get it into the general location of the starting region, andis then configured to automatically acquire grid scans arranged in anoperator-defined vertical and horizontal pattern along either side ofthe airplane fuselage. One of the features of the motion controlalgorithm used here is that it involves distance sensor feedback insteadof requiring individual pre-defined motion paths for the system, whichsimplifies usage and reduces setup time. In order to correctly locatethe scans in the coordinate system of the airplane, 3-D coordinateposition measurements are taken of the boundary regions of the scans. Alocal positioning system is used to acquire 3-D coordinate position datain the coordinate system of the airplane. This reference position datais then used to align the NDI scans with the appropriate airplanecoordinate system.

FIG. 17 is a block diagram identifying some components of a system forinfrared thermographic inspection of large-scale composite structures inaccordance with one computer architecture. Movements of a robot 64 arecontrolled by a robot controller 80 based on a finite-state machine andfeedback from at least distance sensors (e.g., three laser rangemeters). Movements and firing of the LPS 24 are controlled by an LPScontrol computer 48, which also receives laser tracking data from theLPS 24. Activation of the infrared camera 4 and flash lamps 6 arecontrolled by an infrared thermography computer 8, which also receivesinfrared imaging data from the infrared camera 4. All of these computerscan be in wireline or wireless communication with a master computer atan expert workstation 74. The master computer at the expert workstation74 may be programmed to correlate the laser tracking data with theinfrared imaging data. The master computer may be further programmed torequest 3-D model data from a 3-D model database server 96. In the caseof thermographic porosity measurement, the master computer at the expertworkstation 74 may also be programmed to request reference thermalsignature data from a reference thermal signature database server 98.

The LPS control computer 48 acquires location data for the infraredcamera 4 in a 3-D coordinate system of the composite structure. In thecase of a barrel-shaped fuselage section, the infrared imaging data canbe mapped directly onto a 3-D model of the fuselage section. The overlayof infrared imaging data with the 3-D model data enables improved dataanalysis and potential automated data analysis as well. For example,features/flaw indications can be directly correlated to the fuselagestructure by direct overlay of infrared imaging data on the 3-D model.In addition, the direct data overlay onto the model can be used todetermine the thickness of a local area or spatial point, which isneeded for porosity quantification. In one embodiment, the processinvolves application of infrared imaging data strips as one or morecomputer graphics texture maps, which are projected onto the 3-D modelsurfaces in a virtual environment displayed on a monitor or computerscreen at the expert workstation 74.

In some applications, the IRT scanner 214 can be employed with the IRTshroud 216 removed. In cases where the shroud is not needed, additionalequipment for contact-based NDI (e.g., an ultrasonic transducer array oran eddy current probe) can be attached to the IRT scanner 214. Suchembodiments can be enhanced by the provision of means for stabilizingthe distal end of the arm of the automated apparatus. In accordance withvarious embodiments, stabilization is provided by three or morestabilizers each comprising a stationary part and a movable part. Eachstationary part has a fixed location relative to an end effector; eachmovable part is translatably coupled to a respective stationary part andcomprises a contactor disposed at a distal end of the movable part. Whenthe stabilizers are actuated, the contactors are translated toward andinto contact with the surface of the workpiece and then locked in placeto stabilize the distal end of the arm and the end effector coupledthereto. During tool operation, the stabilizers reduce oscillation ofthe end effector 224 (and all structure fixedly coupled thereto).

FIG. 18 is a diagram representing a perspective view of a ground-basedrobotic NDI mobile platform in the process of scanning a curvedworkpiece using a pneumatically stabilized IRT scanner 214 in accordancewith a further embodiment in which an IRT shroud is not employed. Duringthe scanning operation, the end effector 224 and the IRT scanner 214mounted thereto are stabilized by four pneumatic stabilizers 34 a-34 dwhich contact the surface of the curved workpiece 202 in areas outsidethe area to be scanned by the IRT scanner 214. However, the number ofstabilizers can be three instead of four or some number greater thanfour. (If only three stabilizers are employed, they should not becollinear or close to collinear, but rather should be located at thevertices of a triangle that bounds the area of interest.) In theembodiment depicted in FIG. 18, each of the pneumatic stabilizers 34a-34 d comprises a stationary part that is mounted to the housing of theIRT scanner 214. The structure and operation of the pneumaticstabilizers 34 a-34 d will be described in some detail later withreference to FIGS. 20 and 23.

FIG. 19 is a flowchart identifying steps of a method 250 for performinga maintenance operation using a robotic mobile platform equipped with amaintenance tool and a plurality of stabilizers. Method 250 comprisesthe following steps: (a) move a robotic mobile platform to a firstlocation where a tool at a distal end of an arm is able to reach a firstarea of interest on a surface of a target object (step 252); (b)configure the robotic mobile platform such that an end effector coupledto the distal end of the arm is at a correct location relative to thefirst area of interest (step 254); (c) extend contactors of a pluralityof stabilizers into contact with the surface of the target object, whichstabilizers are coupled to the arm (step 256); (d) lock the contactorsof the stabilizers in place to maintain the end effector at the correctlocation relative to the first area of interest (step 258); (e) performa first maintenance operation using the tool while the end effector ismaintained at the correct location relative to the first area ofinterest (step 260); (f) unlock the contactors of the stabilizers uponcompletion of the maintenance operation (step 262); and (g) retract thecontactors of the stabilizers (step 264).

Although not shown in FIG. 19, the method for performing a maintenanceoperation may further comprise: (h) moving the robotic mobile platformto a second location where the tool is able to reach a second area ofinterest on the surface of the target object; (i) configuring therobotic mobile platform such that the end effector is at a correctlocation relative to the second area of interest; (j) extendingcontactors of a plurality of stabilizers into contact with the surfaceof the target object; (k) locking the contactors of the stabilizers inplace to maintain the end effector at the correct location relative tothe second area of interest; and (l) performing a second maintenanceoperation using the tool while the end effector is maintained at thecorrect location relative to the second area of interest. Uponcompletion of the second maintenance operation, the contactors can beagain unlocked and retracted, and the robotic mobile platform can bemoved to the next second location.

In accordance with one proposed implementation, the configuring step ofmethod 250 comprises one or both of the following operations: (a)rotating the end effector so that an axis of the end effector isperpendicular to the surface of the target object in the area ofinterest; and (b) displacing the end effector so that the end effectoris separated from the surface of the target object by a goal offsetdistance. As will be explained in some detail later, a computer systemis configured to control various motors on the automated apparatus independence on feedback from the laser range meters 236, 238 and 240 toachieve a goal orientation of the end effector relative to theconfronting surface of the curved workpiece 202 and a goal offsetdistance separating the end effector from confronting surface of thecurved workpiece 202.

FIG. 20 is a diagram representing a view of a stabilized hybrid toolassembly mounted to an end effector 224 in accordance with analternative embodiment. FIGS. 21 and 22 are diagrams representing frontand side views respectively of the hybrid tool assembly depicted in FIG.20. In this embodiment, the L-shaped mounting plates 218 a-218 c (towhich the laser range meters 236, 238 and 240 are mounted) arerespective links of the end effector 224. The end effector furthercomprises a pair of D-shaped shaft mounting plates 170 (only one ofwhich is visible in FIG. 20) that support respective shaft couplings 172(only one of which is visible in FIG. 20) that couple the end effector224 to a shaft (not shown in the drawings) that is rotatably coupled tothe end of the extension arm 232.

As previously described, an IRT scanner 214 can be mounted to the endeffector 224, which scanner is therefore pivotable about the axis of theaforementioned shaft. The housing of the IRT scanner 214 includes awindow frame 76 having a splash cover attached thereto that forms awindow 332 through which the infrared camera 4 can view the area ofinterest when the robotic NDI mobile platform 200 is being used toperform an IRT inspection.

The hybrid tool assembly depicted in FIG. 22 further comprises anultrasonic transducer array 72 which is movably coupled to the IRTscanner 214 by means of an array support assembly. The IRT shroud 216(see FIGS. 4 and 5) is an enhancement to the base IRT system to allow itto take longer-range images, but the IRT scanner 214 can still be usedto take shorter-range images without the shroud. As will be explained insome detail below, the elements of the array support assembly that mightblock the view of the infrared camera inside the IRT scanner can bemoved to one side out of the way when the IRT scanner 214 is being used

As seen in FIG. 20, the array support assembly comprises a travelingbridge 300 that rides on a pair of mutually parallel horizontal linearrails 312 and 314 attached to the front face of window frame 76. Thetraveling bridge is slidably coupled to the horizontal linear rails 312and 314 by means of respective sliders embedded in block connectors 310and 316. The traveling bridge 300 comprises a pair of mutually parallelvertical linear rails 320 and 322 disposed perpendicular to thehorizontal linear rails 312 and 314. The vertical linear rails 320 and322 are attached to the block connectors 310 and 316. In addition, acarriage 70 is slidably coupled to the vertical linear rails 320 and 322by means of sliders 318 (which are shown at a middle position alongvertical linear rails 320, 322 in FIG. 20 with the carriage removed). Inaccordance with one proposed implementation, the slider/linear railassemblies are caged-ball linear motion guides of a type which arecommercially available from THK Co. Ltd., Tokyo, Japan.

FIGS. 20A and 20B are diagrams representing magnified views ofrespective portions of the hybrid tool assembly within the respectivedashed rectangles 20A and 20B seen in FIG. 20. As shown in FIG. 20A, thetraveling bridge 300 is slidably coupled to the horizontal linear rail312 by means of a slider 328 embedded in block connector 310. As shownin FIG. 20B, the traveling bridge 300 is slidably coupled to thehorizontal linear rail 314 by means of a slider 330 embedded in blockconnector 316.

For the purposes of this disclosure, an X-Y-Z coordinate system will beadopted in which the X direction is parallel to the horizontal linearrails 312 and 314, the Y direction is parallel to the vertical linearrails 320 and 322; and the Z direction is perpendicular to the X and Ydirections. The ultrasonic transducer array 72 is slidably coupled tothe carriage 70 for translation in the Z direction. The carriage 70 inturn is slidably coupled to the vertical linear rails 320, 322 fortranslation in the Y direction. The traveling bridge 300 in turn isslidably coupled to the horizontal linear rails 312, 314 for translationin the X direction.

The array support assembly depicted in FIG. 20 further comprises a pairof lead screws 302 and 304. The lead screw 302 is threadably coupled toa lead screw nut (not shown) incorporated inside block connector 316(see FIG. 20A). The lead screw 304 is threadably coupled to a lead screwnut 338 that is fixedly coupled to the carriage 70 (see FIG. 20B). Oneend of the lead screw 302 is rotatably seated inside a bearing 308 whilethe other end is connected to the output shaft of an X-axis motion motor306. Both bearing 308 and X-axis motion motor 306 are attached to thewindow frame 76. In alternate embodiments, the horizontal linear rails312, 314, bearing 308 and X-axis motion motor 306 may be attached to aframe that is directly mounted to the end effector 224 (in which case,the IRT scanner 214 is removed). In addition, one end of the lead screw304 is rotatably seated inside a bearing 326 while the other end isconnected to the output shaft of a Y-axis motion motor 324. Both bearing326 and Y-axis motion motor 324 are components of the traveling bridge300. The traveling bridge 300 slides along the horizontal linear rails312, 314 whenever lead screw 302 is rotated. The carriage 70 slidesalong the vertical linear rails 320, 322 whenever lead screw 304 isrotated. The direction of translation depends on the direction of leadscrew rotation. Thus the ultrasonic transducer array 72 may be scannedin X and Y directions across the surface of the curved workpiece 202during an ultrasonic inspection procedure.

The array support assembly further comprises a pair of drag chain cablecarriers 174 and 176. The drag chain cable carrier 174 carries anelectrical cable (not shown) that connects to the Y-axis motion motor324; the cable carrier 176 carries an electrical cable (not shown) thatconnects to the ultrasonic transducer array 72. As best seen in FIG.20B, one end of the cable carrier 176 is pivotably coupled to thecarriage 70 and translates vertically in tandem with the carriage 70.The other end of the cable carrier 176 is pivotably coupled to a devicenot shown in FIG. 20A and translates horizontally in tandem with thetraveling bridge 300. Likewise one end of the cable carrier 174 ispivotably coupled to the same device not shown in FIG. 20A andtranslates horizontally in tandem with the traveling bridge 300. Theother end of the cable carrier 174 does not translate.

As best seen in FIG. 20B, the array support assembly further comprises apassive two-axis gimbal 71 which is translatably and rotatably coupledto the carriage 70. The ultrasonic transducer array 72 is mounted to thepassive two-axis gimbal 71, which enables the ultrasonic transducerarray 72 to adjust its angular positions about X and Y axes. The arraysupport assembly further comprises a fifth linear rail 334 disposedparallel to the Z direction and a slider 336, which enable the passivetwo-axis gimbal 71 to translate in a Z direction. The slider 336 isattached to the carriage 70; the linear rail 334 moves with the passivetwo-axis gimbal 71. These features enable the ultrasonic transducerarray 72 to self-correct its location during contact with a curvedsurface by adjusting its position along a Z axis and its pitch and yawangles about the X and Y axes. The passive two-axis gimbal 71 is springloaded to provide compliance in the Z direction. There is no motordriving Z axis motion of the gimbal-mounted ultrasonic transducer array72.

As previously mentioned, the operation of the robotic NDI mobileplatforms disclosed herein and other robotic apparatus having anextended-reach arm can be enhanced by the provision of means forstabilizing the end effector 224 relative to the workpiece. Referring tothe front view of FIG. 21, the stabilization means employed in theembodiment depicted in FIG. 20 includes four pneumatic stabilizers 34a-34 d. Each stabilizer comprises a contactor 154 which is preferablymade of elastomeric material to provide traction and preclude scratchingof the surface of the curved workpiece 202 when the pneumaticstabilizers 34 a-34 d are extended into contact.

As partly shown in FIGS. 20 and 22, each pneumatic stabilizer 34 a-34 dcomprises a pneumatic cylinder 155 which is operatively coupled to apressure regulator 162 by way of a respective solenoid valve 164 and aflexible hose (not shown). The pressure regulator 162 regulates (i.e.,reduces) the pressure of the compressed air being supplied via tubing166, a main air supply 168 and an umbilical cable not shown. Thepneumatic cylinders 155 are preferably of the double-acting type,meaning that they are capable of moving the piston in either one ofopposite directions to produce either an extend stroke or a retractstroke.

More specifically, each pneumatic cylinder 155 comprises a base cylinder158, a piston (not visible in FIG. 22) that is slidable inside the basecylinder 158, and a piston rod 157 connected to the piston and extendingoutside of the base cylinder 158. The contactor 154 is attached to adistal end of the piston rod 157. The states of the solenoid valves 164are controlled by a computer that is configured in a stabilization modeto activate extension of the piston rods 157 by opening the solenoidvalves 164, thereby causing the contactors 154 to move from theretracted position depicted in FIG. 22 to an extended position whereatthe contactors 154 all contact the workpiece surface.

The pneumatic stabilizers 34 a-34 d are compliant at the start of theprocess in order for all four to make contact the surface with aspecific amount of pressure, and they lock in place to keep the assemblyfrom bouncing around while the ultrasonic transducer array 72 is beingmoved on the surface. Thus each pneumatic stabilizer 34 a-34 d furthercomprises a rod lock 160 attached to the base cylinder 158 andconfigured for locking the piston rod 157 to prevent movement. Inaccordance with one proposed implementation, the rod lock 160 preventspiston rod movement upon release of stored energy (i.e., the rod lock160 holds load during power or pressure loss). The rod lock 160 hasdouble-acting locking action for clamping in both directions.

FIG. 23 is a diagram identifying some components of a system forstabilizing a distal end of an extended-reach arm using pneumaticcylinders 155 having rod lock mechanisms 160. The robot control computer80 is configured to receive distance measurement data from the laserrange meters 236, 238 and 240 and then control the various motors of theautomated apparatus to position the end effector 224 in a desiredlocation relative to the area of interest on the workpiece. The robotcontrol computer 80 is also configured to cause the pneumaticstabilizers 34 a-34 d to extend in unison in response to a stabilizecommand from the system operator or in response to a self-generatedstabilize control signal when the desired location has been achieved.The robot control computer 80 is further configured to cause the rodlocks 160 to lock the piston rods 157 in place in response to a lockcommand from the system operator or in response to a self-generated lockcontrol signal when the contactors all contact the workpiece surface.

Although the robot control computer 80 controls the states of all of thepneumatic stabilizers 34 a-34 d, FIG. 23 only shows the components ofone pneumatic stabilizer for the sake of simplicity. The double-actingpneumatic cylinder 158 has two ports to allow compressed air into eitherthe internal volume 186 behind the piston 156 (for the extend stroke ofthe piston rod 157) or the internal volume 188 in front of the piston156 (for the retract stroke of the piston rod 157). Which internalvolume is filled with compressed air from a main air supply 168 isdependent on the state of a cylinder valve 178, which state in turn iscontrolled by the robot control computer 80. The cylinder valve 178 mustbe energized during extension and retraction of the piston rod 157. Itshould also be energized at the end of each stroke until a change ofdirection is required. The pneumatic system further comprises a lockvalve 169 that must be energized during piston motion. When the lockvalve 169 is not energized, the rod lock 160 is engaged.

In contrast to the apparatus depicted in FIG. 22, in which theultrasonic transducer array 72 is movably coupled to an IRT scanner 214,the ultrasonic transducer array 72 can be coupled directly to the endeffector 224 after the IRT scanner 214 has been removed. FIG. 24 is ablock diagram identifying mechanical couplings used to couple anultrasonic transducer array 72 and a stabilizer 50 to the end effector224. For the sake of illustration, it will be assumed that the endeffector 224 is coupled to a distal end 234 of an arm of an automatedapparatus by means of a pivotable coupling 36, meaning that the endeffector is pivotable relative to the distal end 234. In alternativeembodiments, the end effector 224 can be fixedly coupled to the distalend 234. In this example embodiment, the ultrasonic transducer array 72is mounted to a gimbal 71, which gimbal 71 is mounted to a carriage 70that is coupled to the end effector 224 by means of a displaceablecoupling 69, meaning that the carriage 70 is displaceable (e.g., in anX-Y plane) relative to the end effector. In addition, a stabilizer 50 iscoupled to the end effector 224 by means of a fixed coupling 68. Inaccordance with these embodiments, the stabilizer 50 comprises astationary part 52 that is fixedly coupled to the end effector 224 and amovable part 56 that is coupled to the stationary part 52 by means of atranslatable coupling 58, meaning that the movable part 56 istranslatable relative to the stationary part 52. In the exampleembodiment depicted in FIG. 20, the base cylinder 158 and rod lock 160are fixedly coupled to form the stationary part 52 of the stabilizer,while the piston rod 157 is the movable part 56 of the stabilizer.

FIG. 25 is a block diagram identifying components of a control systemthat uses rotation encoders to track the relative location (e.g.,relative to an initial location acquired using a local positioningsystem) of the ultrasonic transducer array 72 depicted in FIG. 20B. Thecontrol system comprises a ground-based control computer 350 programmedwith motion control application software 352 and NDI scan applicationsoftware 354. The control computer 350 is connected to an X-axis motionmotor 360 (which drives translation of the traveling bridge 300 alongthe horizontal linear rails 312, 314) and a Y-axis motion motor 364(which drives translation of carriage 70 along the vertical linear rails320, 322 of the traveling bridge 300). In accordance with one proposedimplementation, the X-axis motion motor 360 and Y-axis motion motor 364are stepper motors that do not require feedback from external encodersfor motion control. The control computer 350 may comprise ageneral-purpose computer programmed with motion control applicationsoftware 352 comprising respective software modules for controlling themotors. The motion control application 352 controls the operation of themotors based on rotation feedback from respective rotational encoders,namely, X-axis rotational encoder 362 and Y-axis rotational encoder 366.The rotational counts from the encoders are converted into linearmeasurements.

The control computer 350 is connected to the motors and encoders via anelectronics box (not shown in FIG. 25). The electronics box contains thesystem power supplies and integrates all the scanner control connectionsand provides an interface between the control computer 350 andrespective flexible electrical cables that connect to the motors. Theencoded data from the X-axis rotational encoder 362 and Y-axisrotational encoder 366 are provided to the ultrasonic pulser/receiver356.

The pulser/receiver 356 sends the encoder pulses to the NDI scanapplication 354. The NDI scan application 354 uses the encoder values toposition the scan data in the proper location.

The control computer 350 hosts ultrasonic data acquisition and displaysoftware that controls the ultrasonic pulser/receiver 356. Theultrasonic pulser/receiver 356 in turn sends pulses to and receivesreturn signals from the ultrasonic transducer array 72. The NDI scanapplication software 354 controls all details of the scan data and thedisplay of data.

FIG. 26 is a block diagram identifying some components of a stabilizercoupled to an end effector 224 by means of a fixed coupling 78 inaccordance with an alternative embodiment. In this example, thestabilizer comprises an outer tube 82 fixedly coupled to the endeffector 224, an inner shaft 86 telescoped inside the outer tube 82, acontactor 88 mounted to a distal end of the inner shaft 86, a spring 84that exerts a spring force that urges the inner shaft 86 to extend untilthe contactor 88 contacts the surface of the workpiece. This type ofspring-loaded stabilizer does not require actuation by a control system.It may be useful in some embodiments to have a locking mechanism.

FIG. 27 is a diagram representing a side view of a ground-based roboticNDI mobile platform having a pneumatically stabilized end effector 224pivotably coupled to the distal end of an arm 232 in accordance with afurther embodiment. In this example, the frame 76 is mounted to the endeffector 224. An ultrasonic transducer array 72 is movably coupled to acarriage 70 in the same manner as previously described with reference toFIG. 20B. The carriage 70 is displaceably coupled to the frame 76 in thesame manner as previously described with reference to FIGS. 20, 20A and21 (only lead screw 304 is shown in FIG. 27 to avoid clutter). Onedifference between the embodiment shown in FIG. 27 and that shown inFIG. 20 is that the four pneumatic stabilizers (only two pneumaticstabilizers 180 a and 180 b are visible in FIG. 27) each comprise apivotable contact pad 182 pivotably coupled to the distal end of thepiston rod 157. Each pivotable contact pad 182 is preferably made ofelastomeric material to provide traction and preclude scratching of thesurface of the curved workpiece 202 when the four pneumatic stabilizers(including pneumatic stabilizers 180 a and 180 b and two additionalpneumatic stabilizers not visible in FIG. 27) are in contact with thatsurface. This embodiment uses contact friction to provide sidewaysmotion resistance, but suction and electrostatic pads could be usedinstead (or even magnetic if the surface is ferromagnetic).

While systems and methods for stabilizing a tool mounted to an endeffector of an extended-reach robotic apparatus have been described withreference to various embodiments, it will be understood by those skilledin the art that various changes may be made and equivalents may besubstituted for elements thereof without departing from the scope of theteachings herein. In addition, many modifications may be made to adaptthe teachings herein to a particular situation without departing fromthe scope thereof. Therefore it is intended that the claims not belimited to the particular embodiments disclosed herein.

As used in the claims, the term “location” comprises position in athree-dimensional coordinate system and orientation relative to thatcoordinate system. As used in the claims, the term “coupled to” shouldbe construed broadly to encompass direct coupling of two components toeach other and coupling of two components by means of interveningcomponents (e.g., a first tool can be coupled to an end effector byattachment to the end effector or by coupling to a second tool or otherintervening component which is attached to the end effector). As used inthe claims, the term “plurality” means three or more.

The methods described herein may be encoded as executable instructionsembodied in a non-transitory tangible computer-readable storage medium,including, without limitation, a storage device and/or a memory device.Such instructions, when executed by a processing or computing system,cause the system device to perform at least a portion of the methodsdescribed herein.

The process claims set forth hereinafter should not be construed torequire that the steps recited therein be performed in alphabeticalorder (any alphabetical ordering in the claims is used solely for thepurpose of referencing previously recited steps) or in the order inwhich they are recited unless the claim language explicitly specifies orstates conditions indicating a particular order in which some or all ofthose steps are performed. Nor should the process claims be construed toexclude any portions of two or more steps being performed concurrentlyor alternatingly unless the claim language explicitly states a conditionthat precludes such an interpretation.

1. An automated apparatus comprising: an arm having a distal end; an endeffector which is coupled to a distal end of the arm; a first tool whichis coupled to the end effector; and a stabilizer comprising a stationarypart and a movable part, wherein the stationary part has a fixedlocation relative to the end effector and the movable part istranslatably coupled to the stationary part and comprises a contactordisposed at a distal end of the movable part.
 2. The automated apparatusas recited in claim 1, wherein the stabilizer comprises a spring.
 3. Theautomated apparatus as recited in claim 2, wherein the stationary partcomprises an outer tube, the movable part further comprises an innershaft slidably disposed inside the outer tube, the contactor is attachedto a distal end of the inner shaft; and the spring urges the inner shaftto translate in a direction so that the contactor moves further awayfrom the end effector.
 4. The automated apparatus as recited in claim 1,wherein the stationary part comprises a base cylinder, the movable partfurther comprises a piston that is slidable inside the base cylinder anda piston rod connected to the piston, and the contactor is attached to adistal end of the piston rod.
 5. The automated apparatus as recited inclaim 4, wherein the stationary part further comprises a rod lockattached to the base cylinder and configured for locking the piston rodto prevent movement.
 6. The automated apparatus as recited in claim 1,further comprising a second tool movably coupled to the first tool,wherein the first tool is fixedly coupled to the end effector, and thestationary part of the stabilizer is fixedly coupled to the first tool.7. The automated apparatus as recited in claim 6, wherein the first toolis an infrared thermography scanner and the second tool is an ultrasonictransducer array.
 8. The automated apparatus as recited in claim 1,wherein the first tool is movably coupled to the end effector, and thestationary part of the stabilizer is fixedly coupled to the endeffector.
 9. The automated apparatus as recited in claim 8, wherein thefirst tool is an ultrasonic transducer array.
 10. The automatedapparatus as recited in claim 9, further comprising: first and secondrails which are disposed parallel to a first direction and at respectivefixed locations relative to the end effector; third and fourth railswhich are disposed parallel to a second direction perpendicular to thefirst direction and translatably coupled to the first and second railsfor travel in the first direction; a carriage which is translatablycoupled to the third and fourth rails for travel in the seconddirection; a fifth rail mounted to the carriage and disposedperpendicular to the first and second directions; and a gimbal which istranslatably coupled to the fifth rail for travel in a third direction,wherein the ultrasonic transducer array is mounted to the gimbal. 11.The automated apparatus as recited in claim 1, wherein the contactorcomprises a pivotable contact pad.
 12. The automated apparatus asrecited in claim 1, wherein the contactor is made of elastomericmaterial.
 13. The automated apparatus as recited in claim 1, furthercomprising a plurality of stabilizers of a type having the structurerecited in claim
 1. 14. A method for performing a maintenance operation,comprising: (a) moving a robotic mobile platform to a first locationwhere a tool at a distal end of an arm is able to reach a first area ofinterest on a surface of a target object; (b) configuring the roboticmobile platform such that an end effector is at a correct locationrelative to the first area of interest; (c) extending contactors of aplurality of stabilizers into contact with the surface of the targetobject, which stabilizers are coupled to the arm; (d) locking thecontactors of the stabilizers in place to maintain the end effector atthe correct location relative to the first area of interest; and (e)performing a first maintenance operation using the tool while the endeffector is maintained at the correct location relative to the firstarea of interest.
 15. The method as recited in claim 14, furthercomprising: (f) unlocking the contactors of the stabilizers uponcompletion of the maintenance operation; (g) retracting the contactorsof the stabilizers; (h) moving the robotic mobile platform to a secondlocation where the tool is able to reach a second area of interest onthe surface of the target object; (i) configuring the robotic mobileplatform such that the end effector is at a correct location relative tothe second area of interest; (j) extending contactors of a plurality ofstabilizers into contact with the surface of the target object; (k)locking the contactors of the stabilizers in place to maintain the endeffector at the correct location relative to the second area ofinterest; and (l) performing a second maintenance operation using thetool while the end effector is maintained at the correct locationrelative to the second area of interest.
 16. The method as recited inclaim 14, wherein the configuring comprises rotating the end effector sothat an axis of the end effector is perpendicular to the surface of thetarget object in the area of interest.
 17. The method as recited inclaim 14, wherein the configuring comprises displacing the end effectorso that the end effector is separated from the surface of the targetobject by a goal offset distance.
 18. A method for performing amaintenance operation, comprising: (a) coupling a tool to an endeffector, which end effector in turn is coupled to a distal end of anarm of an automated apparatus; (b) activating the automated apparatus toperform an alignment operation that locates the tool at a locationrelative to an area of interest on a surface of a target object; (c)extending contactors of a plurality of stabilizers coupled to the distalend of the arm into contact with the surface of the target object whilethe tool is at the location; and (d) activating the tool to perform amaintenance operation while the contactors are in contact with thesurface of the target object.
 19. The method as recited in claim 18,wherein the maintenance operation is non-destructive inspection of aportion of the target object.
 20. The method as recited in claim 18,further comprising locking the contactors of the stabilizers in placewhile they are in contact with the surface of the target object beforeactivating the tool.
 21. An automated apparatus comprising: a baseplatform comprising wheels; an arm movably coupled to the base platformand having a distal end; an end effector which is pivotably coupled tothe distal end of the arm; a tool which is coupled to the end effector;and a stabilizer coupled to the tool or to the end effector andcomprising a rod and a contactor disposed at a distal end of the rod,wherein the rod is selectively extendible or retractable and thecontactor is made of elastomeric material.
 22. The automated apparatusas recited in claim 21, wherein the stabilizer comprises a pneumaticcylinder that is operable to extend or retract the rod.
 23. Theautomated apparatus as recited in claim 22, wherein the stabilizerfurther comprises a rod lock attached to the pneumatic cylinder andconfigured for locking the rod to prevent movement.